Generalized superstatistics of nonequilibrium Markovian systems 
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The paper is devoted to the construction of the superstatistical description for nonequihbrium 
Markovian systems. It is based on Kirchhoff 's diagram technique and the assumption on the system 
under consideration to possess a wide variety of cycles with vanishing probability fluxes. The latter 
feature enables us to introduce equivalence classes called channels within which detailed balance 
holds individually. Then stationary probability as well as flux distributions are represented as some 
sums over the channels. The latter construction actually forms the superstatistical description, 
which, however, deals with a certain superposition of equilibrium subsystems rather then is a formal 
expansion of the nonequilibrium steady state distribution into terms of the Boltzmann type. 

PACS numbers: 02.50.Cw, 02.50.Ey, 02.50.Ga, 05.20.-y, 05.20.Dd, 05.70.Ln 



SUPERSTATISTICS AND STATIONARY STATES 
OF NONEQUILIBRIUM MARKOVIAN SYSTEMS 

During the last several years there has been consid- 
erably grown interest in the description of essentially 
nonequilibrium systems using quasiequilibrium notions, 
namely, the concept of "superstatistics" by Beck & Co- 
hen Briefly, it assumes the stationary state Vf of a 
nonequilibrium system can be written in the Boltzmann 
form with averaging over possible fluctuations in the in- 
verse temperature (3, 



(1) 



fluctuations in the energy dispersion is the standard fact 
going back to early work by Kolmogorov [l^. 

In spite of fundamental problems met in justify- 
ing expression ([1]) the number of papers dealing with 
the superstatistics has increased remarkably in the last 
years. Iii particular, it has been applied to Lagrangian 
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[TtI and Eulerian turbulence [18|, llij 
2l|, atmospheric turbulence [22 



defect turbulence 



cosmic ray statistics 
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Vf^ / d/3/(/3)^exp{-/3i?J, 



2C|, 
23|, 

statistics of solar flares |25| . 
hadronization of quark matter (26| , small- world networks 
p?! , multi-components self-gravitating systems and coUi- 
sionless stellar system s 12811 . transitions between regular- 
chaotic dvnamics |29l. |30| , particle ensembles with frac- 
tional reactions [Sll , analysis of time series 34| , econo- 



physics |33l. 134 



where is the probability distribution of the inverse 
temperature, Ei is the effective energy of the system state 
i, and Z{l3) is the partition function for a fixed value of 
/3. Representation ([1]) is actually a generalization of the 
so-called nonextensive statistics introduced by Tsallis [2] 
and in a integral form relates powerlike and Boltzmann 
distributions [S!]. 

The main idea of superstatistics, however, seems to 
have a longer history at least, expressions similar to 
Eq. ([1]) can be found in monograph by Lavenda and 
this problem goes back to Szilard Q and Mandelbrot @ 
as well as the results of Hungarian school on information 
theory |8|. Moreover, in an effort to derive representa- 
tion ([1]) starting from the general description of statistical 
systems ones have met fundamental problems and incon- 
sistencies HI- On the other hand, the superstatis- 
tical description is rather natural especially for systems 
exhibiting large-scale fluctuations in temperature gradi- 
ents 121 flow turbulence, where the spatiotemporal 



In some sense expression ([T]) can be interpreted in two 
fashions. The first way is to regard expression ([T|) as 
a rather formal expansion of the stationary distribution 
for a nonequilibrium system over terms having the Boltz- 
mann form. In this case the main problem is the rela- 
tionship between the weights /(/?) of this expansion and 
specific random processes governing the system dynam- 
ics, which is currently the main direction of researches 
carried out in this field (see, e.g., Ref. [IBl). The other 
could be an attempt to represent a nonequilibrium sys- 
tem with nonzero stationary probability flux as a cer- 
tain superposition of its subsystems being equilibrium, 
i.e. within which the detailed balance holds individually. 
Exactly this idea is the goal of our study. The purpose 
of the present paper is to implement this approach to 
describing the stationary properties of a nonequilibrium 
Markovian system. Naturally, the flnal integral over pos- 
sible subsystems with local equilibrium has to be of a 
more general form than formula ([1]) and is reduced to 
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it in special cases only, which is the reason of using the 
term "generalized superstatistics" . 

There has been a great deal of studying nonequilibrium 
Markovian systems within the frameworks of the master 
equation, for a review see, e.g., Refs. [s^,!!!,!!^. During 
the last decades many nonequilibrium systems as well as 
systems of other nature where the equilibrium notion is 
irrelevant came into view of physical society. This, in par- 
ticular, has reawaken the interest to the general steady 
state properties exhibited by Markovian systems without 
detailed balance and posed a question about their min- 
imal mesoscopic description called dynamic equivalence 



classes |40l |4l| (see also Ref. [42]). Within the latter 



approach it makes an attempt to find aggregated charac- 
teristics determining the steady state distribution as well 
as probability fluxes in a nonequilibrium system without 
detailed description of all the transition rates between 
the system states. In the present work we actually follow 
the spirit of this idea. 



QUASIEQUILIBRIUM CHANNELS OF A 
NONEQUILIBRIUM MARKOVIAN SYSTEM 

We consider a Markovian system with a finite num- 
ber of states {i}. This number, however, may take any 
large value, so, there should be a feasibility to generalize 
the following constructions to systems with infinite num- 
ber of states. The system evolution is described by the 
master equation 



dt 



(2) 



written for the distribution function Vi, where {i\j) 
stands for the rate of system transitions to state i from 
state j. All the system states make up a graph G whose 
edges present possible transitions between the states. 
Without loss of generality we may adopt two assump- 
tions about this graph, following, e.g., Ref. [s^]. First, 
if there exists a transition from some state j to some 
state i, i.e. > 0, then the reverse transition is also 

possible, i.e. > 0. The opposite case is included 

within the limit 0. Second, the graph G is con- 

nected, which implies that for each pair of states, i.e. 
graph nodes there exists at least one path on 

the graph G (sequence of joint edges) connecting them. 
Otherwise, the physical system behind the graph G can 
be decomposed into two or more independent subsystems 
analyzed individually. 



Kirchhoff's diagrams 

The following part of the paper will deal only with the 
steady state properties of such Markovian systems de- 
scribed by the stationary solution Pi = Vf of the master 



Three state system 
and 

its Kirchhoff's diagrams 




FIG. 1: A three state system and its division into maximal 
trees. 

= io < o j 




FIG. 2: Example of a maximal tree for a six state system and 
its realization for two states, 1 and 5. The correspondence 
between the graph edges and the transition rates is shown 
above. 



equation This solution is represented using Kirch- 
hoff's diagram technique (see, e.g. Refs. [13, E^l). It 
applies to the notion of maximal trees. By definition, 
a maximal tree T for the given graph G is its subgraph 
without cycles that contains all the nodes. Figure [1] illus- 
trates the division of a three state system into its maxi- 
mal trees and Figure [2] depicts some maximal tree T for a 
six state system. Each node i specifies the realization 
of a given tree T orienting its edges in such a way that for 
any node j they make up the path P.^ leading from this 
node j to the node i. Then ascribing the transition rate 
to the directed edge (i, j) the stationary solution of 
equation ([2]) is written as 



(3) 



where Z is the normalization constant (partition func- 
tion) and the effective energy Tii (T) of the state i within 
the tree T is defined as 



(4) 



In particular, for a given tree T the difference Hi (T) 
Hj (T) meets the relation 



n^ (T) - Hj (T) 
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(5) 



for example, for the trees shown in Fig. [2] we have 

(T) - Hi (T) 

= ln[(l|2) (2|4) (4|5)]-ln[(5|4) (4|2) (2|1}] . 

The steady state probability flux J.^ through the edge 
{ij) from the node j to the node i is, by definition, := 
{i\j) Pj — {j\i) Pi. So by virtue of ([3]) the equality 



It. Mm n (^1^) 



holds. Here C{(i,j),T} is the cycle created via con- 
necting the node j to the node i and its forward trac- 
ing is given by the transition from j to i, the set T.y = 
T \ C {{«, j), T} is the collection of subtrees remaining af- 
ter the edges of cycle C{(i,j),T} having been removed 
from the tree T, and 



(fe06C+{(ij),T} 



n (^10 

ikl)eC-{ii,j).J} 



(7) 

is the partial probability flux through the edge re- 
lated to the cycle C{(i,j),T}, where the superscripts + 
and — label the forward and backward directions of the 
cycle tracing. We point our that the quantity Jc{{i,j).T} 
as well as the total probability flux Jy is antisymmetric 
with respect to the index interchange, whereas the set Ty 
and, as a consequence, its contribution to the flux Jy re- 
main the same under this transformation. The items in 
sum ^ are depicted by the diagram in Fig. [3l The exis- 
tence of nonzero probability flux under the steady state 
conditions is actually the manifestation of the system be- 
ing nonequilibrium and detailed balance not holding. 



Three state system. Illustrating example 

To illustrate Kirchhoff's technique described above we 
consider the three state system shown in Fig. [T] and imi- 
tating particle hopping in an random medium. Without 
some driving field £ the system is assumed to be equilib- 
rium one with the detailed balance, which is reduced to 
the equality 

(1|2) (2|3) (3|1) = (1|3) (3|2) (2|1) , 



C{(z,j),T} 



X 




T\C{(»,j),T} 



FIG. 3: Diagram visualizing an item in sum (|6]). Here 
C{(i,j),T} is the cycle created by connecting the nodes j, 
i of the minimal tree T, where the forward direction of cycle 
tracing is given by transition from the node j to the node i. 



implying the absence of the probability flux along the 
cycle 1-2-3-1. The particle hopping between potential 
traps {uj} is described by the transition rates 

where oj is some kinetic coefficient. The field £ breaks the 
detailed balance disturbing the transition rates as follows 

(l|2)^ = (l|2)e^ (2|l)^ = (2|l}e-^ 



(2|3)^ = (2|3)e^ 
(3|l)^-(3|l)e^ 



(3|2)£-(3|2}e- 
(l|3)£ = (l|3)e^ 



Then using the general expression ([3]) for the stationary 
distribution and applying to Fig. [1] showing the corre- 
sponding collection of maximal trees we get the expres- 
sion for the partition function 

Z{£) = cj2e-"i-«2-"3 (gui ^ ^ gU3) + 2cosh(2£:)] . 

Then by virtue of ([6]) the probability flux along any edge 
of this system is 

2uj sinh £ 



J ^ 



(e"i + e«3 -I- e«3) 



Below the effective energy Tii (T) for the state i within 
a given tree T has been introduced by expression Q. 
For the three state system under consideration applying 
to Fig. [T] we can write, for example. 



H2(Ti) 


-Hi(Ti) 


= 2£- 


{U2 - 


Ul) , 


W3(Ti) 




= 45- 


("3 - 


Ul) , 


H2(T2) 


-Hi(T2) 


= 25- 


("2 - 


Ul) , 


W3(T2) 


-Hi(T2) 


= ~2£ 


- {U3 


- Ui 



Whence it follows that the effective energies 7ii(Ti), 
'Hi{T2) of the trees Ti, T2 cannot be represented in 
terms of one energy H^'^ multiplied by some individual 
cofactors /3i, (32, i-e. there is no function H^" such that 
H,(Ti) ^ PiHI" and Hi(T2) ^ P2H.r . Indeed, other- 
wise, the relations (3i = /32 and (3i ^ (32 have to hold 
simultaneously. 
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FIG. 4: Illustration of the cycle formation by the superposi- 
tion of maximal trees. 



Equivalence classes of maximal trees. Channels 

At the next step a special case of Markovian systems 
is analyzed, where there are collections of many maximal 
trees within which the detailed balance holds individu- 
ally. Namely, let us consider some two maximal trees T" , 
T'^ , and the union of their edges T" IJ T'^ referred below 
as to the tree superposition. The two trees are said to 
be equivalent, T" ^ T^, if their superposition does not 
contain any cycle C of edges with nonzero probability 
flux Jc ^ 0, which is illustrated in Fig. IH Applying, for 
example, to the cycle C shown in Fig. |4] the condition of 
zero probability flux reads 

m U\k2) {k2\ki) (fcilz) = (fcilfca) (felj) {M 



thus 



{j\k2) {k2\ki) _ 



(zifci)(fciifc2)(fc2U) m 

whence, by virtue of ©, we get the identity 

This example demonstrates the fact that for equivalent 
trees, e.g., T"and T'' their effective energies differ in con- 
stant value only, with latter statement being actually an 
equipollent definition of the tree equivalence, 



const . 



(8) 



Thereby the introduced relationship between the maxi- 
mal trees really form an equivalence relation because the 
condition T" ~ and T" ^ T"*" gives rise to T'' ~ T'''. 
Therefore the collection of all the maximal trees {T} of 
the state graph G can be divided in the classes {K} of 
equivalent trees that will be called channels. The super- 
position of all the maximal trees belonging to one channel 
K i.e. its implementation as a subgraph of graph G will 
be also referred to as just the channel K. 

The fact that the notion of channels does have some 
meaning is justified in Fig. [5] exhibiting a case where there 
are channels contaning more then one maximal tree and 



not coinciding with the system as a whole. Namely, this 
figure depicts a four state system for which there are two 
elementary (three state) cycles with zero probability flux 
(cycles 1-3-4-1 and 2-3-4-2) and two ones with nonzero 
flux (cycles 1-3-2-1 and 1-2-4-1). These cycles, indeed, 
can have such properties, which requires some comments. 
The matter is that the fluxes Ji_3_2-i and Ji_2-4-i are not 
independent because the compound cycle 1-3-2-4-1 can 
be obtained unifying either the cycles 1-3-2-1 and 1-2-4-1 
or the cycles 1-3-4-1 and 2-3-4-2. The unification of cy- 
cles with zero probability flux inevitably gives rise to the 
compound cycle with zero flux too. So the fluxes Ji-3-2-1 
and Ji_2-4-i should also meet the condition of zero prob- 
ability flux for the cycle 1-3-2-4-1. The detailed analysis 
of the relationship between the probability fluxes of ele- 
mentary and compound cycles is beyond the scope of the 
present paper. Here we note only that the adopted flux 
pattern can be implemented for the given graph when 
all its edges are symmetrical with respect to the transi- 
tion rate, = except for the edge (1,2), where 
(1|2) ^ (2|1). The induced partition of the maximal tree 
collection into the channels and their implementation in 
graph form are shown in Fig. [5l 



Channel superstatistics of the steady state 
distribution 



In order to describe the properties of a maximal tree 
T within its channel K containing TYk trees, first, the 
effective energy Tii {K} averaged over the given channel 



T'6K 



(9) 



is introduced. The quantity Hi (K) will be called the 
effective energy of the state i within the channel K. Then, 
using the constructed function the effective energy Tii (T) 
of the state i within the tree T in rewritten as 

7^, (T) = i/K(j) + (7 (T|K) , (10) 

where the value 

^=^E[^'^(T')-W.(T)] (11) 



;7(T|: 



T'eK 



is regarded as the effective energy of the tree T within 
the channel K because due to property ([8]) it does not 
depend on the state i and, thus, characterizes the tree T 
as a whole. Naturally, the mean value of [/ (T|]K) is equal 
to zero. 



(12) 



Tel 



The introduced quantities permit us to rewrite expres- 
sion ^ for the steady state distribution of the given 
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FIG. 5: An example of four state system and its division into 
channels I-V. In addition the elementary cycles with zero and 
nonzero probability flux are shown. 



Markovian system as a sum running over the channels 
P^^\Y. ^Ke-^'' exp {-Bull)) 
= i^u;(K)exp{-iJK(*)} • 



(13) 



Here the quantity Fjj has appeared in formula (|T3|) via 
the sum over all the maximal trees {Tjjj composing the 
channel K 

exp(-^k) :-^5Ie^P{-^(T|K)} (14) 

and specifies the statistical weight of channel K 
w;(IK) = TYKe-^"^ . 



In order to write the desired expression for _Fk we make 
use of the expansion 

e"'^ = 1 - f7 + ^{\J) , where ip<yU) > for [/ 7^ . 



Whence, by virtue of ([T 



exp (-Fk) = 1 



1 



E 



^[?7(T|K)] ='exp{0K5K} 



(15) 

where Sk '■= IuA'k has the meaning of channel entropy. 
Due to the zero value of the mean channel energy Vk = 
the quantity Fe, can be rewritten as i^K = Mk ~ 0kSk 
and keeping in mind the standard notions of statistical 
physics it will be called the free energy of channel K. In 
particular, the quantity 



Ok 



1 



(16) 



is an order parameter of the channel structure; when all 
the trees of a channel K have the same energy J7 (T|K) = 
U (K) the value = and the wider the distribution of 
the tree energies, the large the value 9k- 

Finalizing this section we rewrite formula psp as 



P^ = ^^NK exp {9kSk - HK{i)} 



(17) 



Expression (fT7|) is the desired superstatistics representa- 
tion of the steady state distribution for a nonequilibrium 
Markovian system. It should be noted that the obtained 
effective energy i?K(i) of the system states {i} within the 
channel IK can depend on many parameters of this chan- 
nel and is reduced to some fixed function H{i) multiplied 
by an inverse channel "temperature", i?K(«) — (3KH{i), 
in special cases only. This could take place if, for exam- 
ple, the distribution of the tree energies within one chan- 
nel and the distribution of the system states are caused 
by the same mechanism. In this case it might be ex- 
pected that the channel temperature will be specified by 
the order parameter 0k, i.e. 1/Pk — 9k- 

Figure [5] visualizes an example of systems where dif- 
fcrcn channels, at least some ones, can be ascribed with 
the effective energies for the system states that differ only 
by some constant prefactors. It is a stair case system with 
the transition rates shown in Fig. [SI where the lower two 
diagrams depict equivalent maximal trees. Rigorously 
speaking, these trees become equivalent only on scales 
5x ^ a 01, what is actually the same, in the limit a — > 0. 
Indeed, at the first approximation the energy of a state 
i located near the point x along the system (Fig. (S]) can 
be written as 



Hk{x) = '2{qu£u 



a 



const , 



where qu^i is the relative portion of edges on the upper 
and lower branches belonging to the trees of a given chan- 
nel, qu-\-qi = \. In some sense the value q^ (or q;, what is 
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maximal trees 



FIG. 6: Example of a system where at least some channels 
can be characterized by the effective energies of the states 
differing only by constant prefactors. 



the same) is the main characteristics of such a channel. It 
is assumed to be distributed rather uniformly along the 
system because exactly the uniform distribution matches 
the maximal number of the channel realizations in the 
maximal trees. Naturally, there are channels in this sys- 
tem where the portion Qu and, thus, also qi can depend on 
the coordinate x and, thereby, their effective energies are 
not reduced to the presented form. However, their con- 
tribution to the steady state distribution could be rather 
small. 



Channel superstatistics of the steady state 
probability flux 



The steady state probability flux {Jij} through the 
graph edges can be also reduced to a sum over the chan- 
nels. In order to do this we make use of formula ([6]). 
Let us consider an edge and two trees T", T'^ e K 
belonging to one channel K. The given edge is as- 
sumed beforehand not to belong to the channel K be- 
cause otherwise all its maximal trees do not contribute 
to the flux Jij. The edge with the edges of the 



trees T" and form two cycles C" = [j^ij and 
= {hi)[J^i.j^ where Ff^^, P^^- are paths on the 
trees T", T*^, respectively, connecting the nodes i and j 
(Fig. [7|) . The paths P"^ , P^j can coincide with each other 
partly or even completely. Since the two trees belong to 
one channel the probability flux along the composite cy- 
cle C"/^ = c" u C'^ - p^j U J 



latter condition is implemented by the equality 

n (^10 n (^10= n (^lo n (^i^)- 

(18) 

in particular, for the case shown in Fig. [7| 

(i|ai)(ai|«2)(a2|z) (*|/32) (/32|/3i) (/3i|j) 

= im) {Pim mi) (*|a2) (a2|«i) . 

Let us introduce the intensities Rfj, i?^ , and the asym- 
metry Aij (K) of the transitions along the paths ¥fj , Pf ^■ 
via the expressions 



,a,/3 



R 



n (^10 n (^10 



n (^10 



1/2 



n (^10 

(/ci)eP'3+ 



- 1/2 






n (^10 






- 1/2 






n (^10 




(ki)erf<- 



(19) 

-1/2 
-1/2 

(20) 



with the value Aij (K) being the same for both the paths 
P^j, Pf^ due to (HH). In other words, the quantity 
Aij (K) is the characteristics of the edge with re- 

spect to the channel K rather than to its trees individ- 
ually. It should be pointed out that the quantities Rfj 
are symmetrical whereas Aij (K) is asymmetrical with 
respect to the index interchange, i.e., R]j — Rjj^ and 
Aij (K) = —Aji (K). In addition, let the quantities 

and Pfj stand for the contributions of the subtree collec- 
tions T" \ C" and T'^ \ (Fig. [3|) to the tree weights 
exp {— (T")} and exp |— Hi (T'^)}, respectively. As 
noted above the quantities P^j are symmetrical within 
the index interchange. Applying to definition ([4]) of the 
effective energy Hi {T"^) we write (7 = a, /?) 



exp{-W,(Tr^)} = 
exp{-W,(T^)} = 



i?TP^.exp{-A,(]K)} 
i?7^.P^.exp{A,(K)} , 



whence taking also into account relationship pH)) be- 
tween the effective energies of the system state i within 
the tree T''' and the corresponding channel K we get 



1 



i?7,P^. = exp --[HaT^) + W,(T 



;ljpf is equal to zero. The 



e^p\~-[HK{i)+HKij)]-UiT->\ 



In these terms the partial probability flux (see (O and 
(O) for the two trees T°, T'' individually becomes (7 = 



7 




FIG. 7: Two cycles C" and created via the connection of 
the nodes i and j on the maximal trees T'* and T'' belonging 
to one channel K. Arrows specify the forward direction of the 
cycle tracing. 

a, 13) 

{T') = Jc{(ij),T^} n ^^1^^ 

= exp {A, m - m exp {-A, (K)}] ^P^^ 
= (K) exp |-i [HK{i) + Hk{j)] - U (T^|K)| , 

(21) 

where the quantity £ij (K) introduced by the expression 

(K) exp {A, (K)} - exp {^A,, (K)}] 

(22) 

is ascribed directly to the edge with respect to the 
channel K. It characterizes the contribution of the chan- 
nel K to the stationary probability flux through the edge 
(«, j), namely, by virtue of ®, HI]), and 

= I E (^) ^Ke-^*' exp |-i [i7K(*) + HkU)]] . 

(23) 

Expression (|23|) is the desired superstatistics represen- 
tation of the stationary flux of probability. In some 
sense the nonequilibrium properties of the Markovian 
system under consideration are behind the driven forces 
{£ij (K)} induced by channel K. In particular, if the edge 
belongs to the channel K then £ij (K) = 0. 

CONCLUSION 



the form of the probability distribution as a weighted 
sum over equilibrium distributions up to now is based 
on purely phenomenological arguments. No fundamental 
derivation from the basic principles has been given. 

In the present paper we have been able to derive an 
analogy to the superstatistics approximation with respect 
to the probability distribution as well as the stationary 
probability fluxes. This means that superstatistical rep- 
resentations can also be formulated for fluxes, which has 
not been discussed up to now. Our treatment is based 
on Kirchhoff's diagram technique applied to the master 
equation for Markovian processes. This techniques has 
been known for many years. We have included an as- 
sumption on the properties of the system graphs. As a 
main point we have focused on systems possessing a wide 
collection of cycles with vanishing fluxes. In this case it 
is possible to construct equivalence classes of Kirchhoff's 
maximal trees called channels, for which detailed bal- 
ance holds individually. The nonequilibrium properties 
initially attributed to cycles can be assigned to individ- 
ual edges, i.e. to individual transitions between a pair of 
states within one channel. 

It is worthwhile to underline the fact that in the 
present paper we actually have formulated an original 
approach to describing steady states of nonequilibrium 
systems. In its spirit it is rather similar to the widely 
used notion of superstatistics but, nevertheless, differs 
from at the basics. The key point of the developed ap- 
proach is the representation of a nonequilibrium system 
as a superposition of its equilibrium subsystems (chan- 
nels) with local detailed balance rather then a formal 
expansion of the steady state distribution into the sum 
over terms of the Boltzmann type. Our considerations 
shed light on the network structures of complex systems 
and may help to understand systems in flux equilibrium 
like turbulent flows, traffic flows and economic systems. 
Beside, the introduced notion of channels enables one to 
pose a question as to whether it is possible to describe the 
dynamics of Markovian systems as transient processes in 
the channels individually and their interaction with one 
another. 
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In conclusion we have derived a general form of the sta- 
tionary probability distribution for Markovian systems. 
This has allowed us to obtain a novel interpretation of 
probability distributions used in the framework of su- 
perstatistics, or more rigorously, generalized superstatis- 
tics. This notion has been successful in reproducing the 
probability distribution of many systems ranging from 
turbulence to economics. However, the justification of 
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